The Case for Building High Performance Intermediate-Level Memory Systems

This paper presents the architecture of a high-performance intermediate-level memory subsystem. The memory subsystem is designed to narrow the growing gap between processors’ cycle time and the round-trip delay to main memory. The system is built from high-performance DRAM arrays and SRAM buffers, both integrated on the same IC. We call these parts Integrated Static and Dynamic Random Access Memory (ISDRAM). The ISDRAM system is configured as a very large cache and can be implemented either as an on-chip cache integrated with the CPU, or as a much larger external cache. We show that building very large caches can be an effective way to narrow the growing CPU-memory speed gap. Building a cache as a combination of DRAM and SRAM yields a system with a relatively low cost per bit while achieving performance that is close to that of a system with a very large cache made of (expensive) fast SRAM. We demonstrate that ISDRAM cache, containing SRAM buffers that are only 1/64th the size of DRAM, performs almost as well as a conventional SRAM cache of the same size. “Back of the envelope” estimates show that the ISDRAM bit-density should be approximately 1/2 the bit-density of commercial DRAM parts and at least 5x denser than a conventional on-chip SRAM cache of the same size. 1.0 The Case for High Performance Intermediate-Level Memory System The CPU-memory speed gap poses a considerable performance barrier. To illustrate the problem, 9 SPEC benchmarks were selected. SimpleScalar processor [2] with a 256KB L2 cache (a baseline configuration) running at 2Ghz, 4GHz and 8GHz was simulated executing these benchmarks. A main-memory model with delays independent of the processor clock frequency was used. All other delays (including L1 and L2 cache delays) were modeled in numbers of clock cycles. Simulation details are given in section 4. For each benchmark, the time (#_clocks x clock_period) it took the processor to execute a fixed number of instructions using different clock frequencies, was used to estimate the fraction of time the processor is idle waiting on memory requests. The computation is as follows: We assume that the delay is a linear function of the clock period, that is, the time it takes to execute N instructions T=A*t + B, where t is the clock period. The constant A capture the system’s frequency dependent component and B is the clock independent component, namely B=Idl-time. This is only an approximation, since a superscalar processor is a complex dynamic system that is sensitive to relative component delays. However, checking the model accuracy by calculating the discrepancy between the linear model and simulated execution times show the error to be small (~1%). If T1 is the time it takes to execute N instructions at one frequency, and